{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Docker提交\n",
    "\n",
    "本次竞赛的Docker提交大致可以分为两小块：\n",
    "\n",
    "1. 线下文件准备好：包括DockerFile，代码，预测的代码；\n",
    "2. Build同时pull提交\n",
    "\n",
    "如果之前没有提交过docker，可以根据这篇教程熟悉一下：https://tianchi.aliyun.com/forum/postDetail?spm=5176.12586969.1002.9.51df4127FoZKeL&postId=165595 \n",
    "\n",
    "\n",
    "## 线下文件准备\n",
    "\n",
    "### Requirement \n",
    "\n",
    "- 运行代码所依赖的python库，缺什么就把需要装的文件放在requirement下面"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "numpy\n",
    "tensorflow==2.2.0 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 运行的代码\n",
    "\n",
    "#### 放在code下面即可"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow.keras.backend as K\n",
    "from tensorflow.keras.layers import *\n",
    "from tensorflow.keras.models import *\n",
    "from tensorflow.keras.optimizers import *\n",
    "from tensorflow.keras.callbacks import *\n",
    "from tensorflow.keras.layers import Input \n",
    "import numpy as np\n",
    "import os\n",
    "import zipfile\n",
    "\n",
    "def RMSE(y_true, y_pred):\n",
    "    return tf.sqrt(tf.reduce_mean(tf.square(y_true - y_pred)))\n",
    "\n",
    "def build_model():  \n",
    "    inp    = Input(shape=(12,24,72,4))  \n",
    "    \n",
    "    x_4    = Dense(1, activation='relu')(inp)   \n",
    "    x_3    = Dense(1, activation='relu')(tf.reshape(x_4,[-1,12,24,72]))\n",
    "    x_2    = Dense(1, activation='relu')(tf.reshape(x_3,[-1,12,24]))\n",
    "    x_1    = Dense(1, activation='relu')(tf.reshape(x_2,[-1,12]))\n",
    "     \n",
    "    x = Dense(64, activation='relu')(x_1)  \n",
    "    x = Dropout(0.25)(x) \n",
    "    x = Dense(32, activation='relu')(x)   \n",
    "    x = Dropout(0.25)(x)  \n",
    "    output = Dense(24, activation='linear')(x)   \n",
    "    model  = Model(inputs=inp, outputs=output)\n",
    "\n",
    "    adam = tf.optimizers.Adam(lr=1e-3,beta_1=0.99,beta_2 = 0.99) \n",
    "    model.compile(optimizer=adam, loss=RMSE)\n",
    "\n",
    "    return model \n",
    "\n",
    "model = build_model()\n",
    "model.load_weights('./user_data/model_data/model_mlp_baseline.h5')\n",
    "\n",
    "test_path = './tcdata/enso_round1_test_20210201/'\n",
    "\n",
    "### 1. 测试数据读取\n",
    "files = os.listdir(test_path)\n",
    "test_feas_dict = {}\n",
    "for file in files:\n",
    "    test_feas_dict[file] = np.load(test_path + file)\n",
    "    \n",
    "### 2. 结果预测\n",
    "test_predicts_dict = {}\n",
    "for file_name,val in test_feas_dict.items():\n",
    "    test_predicts_dict[file_name] = model.predict(val).reshape(-1,)\n",
    "#     test_predicts_dict[file_name] = model.predict(val.reshape([-1,12])[0,:])\n",
    "\n",
    "### 3.存储预测结果\n",
    "for file_name,val in test_predicts_dict.items(): \n",
    "    np.save('./result/' + file_name,val)\n",
    "\n",
    "#打包目录为zip文件（未压缩）\n",
    "def make_zip(source_dir='./result/', output_filename = 'result.zip'):\n",
    "    zipf = zipfile.ZipFile(output_filename, 'w')\n",
    "    pre_len = len(os.path.dirname(source_dir))\n",
    "    source_dirs = os.walk(source_dir)\n",
    "    print(source_dirs)\n",
    "    for parent, dirnames, filenames in source_dirs:\n",
    "        print(parent, dirnames)\n",
    "        for filename in filenames:\n",
    "            if '.npy' not in filename:\n",
    "                continue\n",
    "            pathfile = os.path.join(parent, filename)\n",
    "            arcname = pathfile[pre_len:].strip(os.path.sep)   #相对路径\n",
    "            zipf.write(pathfile, arcname)\n",
    "    zipf.close()\n",
    "make_zip() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### run.sh\n",
    "\n",
    "- 运行预测的代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!/bin/sh\n",
    "CURDIR=\"`dirname $0`\" #获取此脚本所在目录\n",
    "echo $CURDIR\n",
    "cd $CURDIR #切换到该脚本所在目录\n",
    "python /code/mlp_predict.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DockerFile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Base Images\n",
    "## 从天池基础镜像构建 \n",
    "FROM registry.cn-shanghai.aliyuncs.com/tcc-public/tensorflow:latest-cuda10.0-py3\n",
    "\n",
    "## 把当前文件夹里的文件构建到镜像的根目录下（.后面有空格，不能直接跟/）\n",
    "ADD . /\n",
    "\n",
    "## 指定默认工作目录为根目录（需要把run.sh和生成的结果文件都放在该文件夹下，提交后才能运行）\n",
    "WORKDIR /\n",
    "\n",
    "## Install Requirements（requirements.txt包含python包的版本）\n",
    "## 这里使用清华镜像加速安装\n",
    "RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --upgrade pip\n",
    "RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt\n",
    "#RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt\n",
    "\n",
    "## 镜像启动后统一执行 sh run.sh\n",
    "CMD [\"sh\", \"run.sh\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 其它\n",
    "\n",
    "- 按照官方要求把所需的文件全部按要求准备好即可。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线上提交\n",
    "\n",
    "在所有的文件都准备之后，下面一步就是进行线上的提交，这里又分为三块。\n",
    "\n",
    "1. 按照要求进行线上配置\n",
    "2. 进行build和pull；\n",
    "3. 提交"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 按照要求进行线上配置\n",
    "\n",
    "<img src=\"./pic/first_step.jpeg\" width = \"500\" height = \"200\" alt=\"first_step\" align=center />\n",
    "\n",
    "\\\\ \n",
    "\n",
    "<img src=\"./pic/second_step.jpeg\" width = \"500\" height = \"200\" alt=\"first_step\" align=center />\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 进行build和pull；\n",
    "\n",
    "\n",
    "<img src=\"./pic/1st.jpeg\" width = \"400\" height = \"200\" alt=\"first_step\" align=center />\n",
    "\\\\\n",
    "<img src=\"./pic/2nd.jpeg\" width = \"400\" height = \"200\" alt=\"first_step\" align=center />\n",
    "\\\\\n",
    "<img src=\"./pic/3rd.jpeg\" width = \"400\" height = \"200\" alt=\"first_step\" align=center />\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### 1.登录\n",
    "sudo docker login --username=\"自己的用户名\" registry.cn-shenzhen.aliyuncs.com\n",
    "#### 2.build\n",
    "docker build registry.cn-shenzhen.aliyuncs.com/ai_earth_baseline/test_ai_earth_submit:1.0 .\n",
    "#### 3.push\n",
    "docker push registry.cn-shenzhen.aliyuncs.com/ai_earth_baseline/test_ai_earth_submit:1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 提交\n",
    "\n",
    "\n",
    "<img src=\"./pic/submit.jpeg\" width = \"600\" height = \"200\" alt=\"first_step\" align=center />\n",
    "\n",
    "根据自己的不同进行提交即可，如果不出意外，等待一会儿，线上跑完了就会有结果了。\n",
    "\n",
    "<img src=\"./pic/score.jpeg\" width = \"600\" height = \"200\" alt=\"first_step\" align=center />\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "288px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
